With the increasing popularity of playing streaming audio and video over networks such as the internet, there is a need for optimizing the data transferred from a server to a client such that the client's experience is maximized even if network conditions during playback are inconsistent. Optimizing the client's experience involves choosing a quality level for encoding the video portions of the video playback such that the video can be transferred and reconstructed uninterrupted while preserving the quality of the video content.
The quality level is generally dictated by the bit rate specified for the encoded video portions of the input stream. A higher bit rate generally indicates that a larger amount of information about the original video is encoded and retained, and therefore a more accurate reproduction of the original input video will be presented during video playback. Conversely, a lower bit rate indicates that less information about the original input video is encoded and retained, and thus a less accurate reproduction of the original video will be presented during video playback.
Generally, the bit rate is specified for encoding video based on several factors. The first factor is the network condition between the server and the client. A network connection that can transfer a high amount of data indicates that a higher bit rate can be specified for the input video that is subsequently transferred over the network connection. The second factor is the desired start-up latency. Start-up latency is the delay that a video playback tool experiences when first starting up due to the large amount of data that has to be received, processed, and buffered. The third factor is the tolerance to glitching. Glitching is when video playback has to stop because data is missing. In most cases any amount of start-up latency or glitching is intolerable, and it is therefore desirable to optimize the bit rate specified such that the start-up latency and the glitching are minimized or eliminated.
Currently available commercial streaming media systems rely on multi bit rate (MBR) coding to perform coding rate control. In MBR coding, source video content is encoded into alternative bit streams at different coding rates and typically stored in the same media file at the server. This then allows the content to be streamed in segments or chunks at varying levels of quality corresponding to different coding rates according to the changing network conditions, typically using bit stream switching between segments.
Transcoding is another technique that can be used, wherein rather than storing content at different encoding rates, the content is transformed from one compressed format to another in real-time, while it is being transmitted over the network. Real-time transcoding in a many-to-many way (any input format to any output format) is becoming a necessity to provide true search capability for any multimedia content on any mobile device, with over 500 million videos on the web and a plethora of mobile devices.
New techniques for improving video quality while transcoding are constantly in demand.